GENERALIZED SERRE DUALITY 



XIAO-WU CHEN 



Abstract. We introduce a notion of generalized Serre duality on a Hom-finite 
Krull-Schmidt triangulated category T. This duality induces the generalized Serre 
functor on T, which is a linear triangle equivalence between two thick triangulated 
subcategories of T. Moreover, the domain of the generalized Serre functor is the 
smallest additive subcategory of T containing all the indecomposable objects which 
appear as the third term of an Auslander-Reiten triangle in T; dually, the range of 
the generalized Serre functor is the smallest additive subcategory of T containing 
all the indecomposable objects which appear as the first term of an Auslander- 
Reiten triangle in T ■ 

We compute explicitly the generalized Serre duality on the bounded derived 
categories of artin algebras and of certain noncommutative projective schemes in 
the sense of Artin and Zhang. We obtain a characterization of Gorenstein algebras: 
an artin algebra A is Gorenstein if and only if the bounded homotopy category of 
finitely generated projective A-modules has Serre duality in the sense of Bondal 
and Kapranov. 

1. Introduction 

Throughout R will be a commutative artinian ring. Let T be an i?-linear trian- 
gulated category which is Hom-finite and Krull-Schmidt. Here, an iZ-linear category 
is Hom-finite provided that all its Horn spaces are finite generated i?-modules; an 
additive category is Krull-Schmidt provided that each object is a finite sum of inde- 
composable objects with local endomorphism rings. 

The Auslander-Reiten theory for a Hom-finite Krull-Schmidt triangulated category 
T was initiated by Happel ( [12\ 113]) which now plays an important role in the repre- 
sentation theory of artin algebras. The central notion is Auslander-Reiten triangle, 
which by definition is a distinguished triangle X A Y A Z X[l] with X, Z inde- 
composable such that u (resp. v) is left {resp. right) almost-split. Here [1] denotes the 
translation functor on T. For an indecomposable object X there exists at most one, 
up to isomorphism, Auslander-Reiten triangle which contains X as the first term; 
dually, for an indecomposable object Z there exists at most one Auslander-Reiten 
triangle which contains Z as the third term. The triangulated category T is said to 
have enough Auslander-Reiten triangles provided that for each indecomposable ob- 
ject X there exist two Auslander-Reiten triangles which contain X as the first term 
and the third term, respectively. A fundamental result due to Reiten and Van den 
Bergh states that T has enough Auslander-Reiten triangles if and only if it has Serre 
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duality ( \26\ Theorem 1.2.4]). Here the Serre duality is in the sense of Bondal and 
Kapranov (J7J). 

Observe that in general a Horn-finite Krull-Schmidt triangulated category T will 
not have enough Auslander-Reiten triangles. For example, Happel proved that the 
bounded derived category of an artin algebra has enough Auslander-Reiten triangles 
if and only if the algebra has finite global dimension (|13|). In this situation, the 
category T will not have Serre duality. 

In the present paper, we introduce a notion of generalized Serre duality on a Horn- 
finite Krull-Schmidt triangulated category T ■ This generalized duality induces the 
generalized Serre functor of T, which is a linear triangle equivalence between certain 
thick triangulated subcategories of T ■ Here, thick subcategories mean subcategories 
which are closed under direct summands. We put Reiten-Van den Bergh's theorem in 
this general setting. It turns out that this generalized Serre duality is closely related 
the Auslander-Reiten triangles in T ■ 

The notion of generalized Serre duality applies to an arbitrary Horn-finite i?-linear 
category. We need some notation. Let C be a Horn-finite ii-linear category. For two 
objects X and Y, we write (X, Y) for the Horn space Home(X, Y). Denote by (X, — ) 
(resp. (— ,X)) the representable functor Homc(X, —) (resp. Homc(— ,X) ) from C 
to the category i?-mod of finitely generated -R-modules. Denote by D the Maths 
duality on i?-mod. Recall that D = Hom/j(— ,E) where E is the minimal injective 
cogenerator for R; see Chapter II. 3]. By D(X, — ) (resp. D(—,X)) we mean the 
composite functor DHomc(X, — ) (resp. DHome(— , X)). 

We define two full subcategories of C as follows 

C r = {X € C | D(X, — ) is representable} and C\ = {X € C \ D(—,X) is representable}. 

It turns out that there exists a unique functor S: C r — > C\ such that there is a i?-linear 
isomorphism 

<t> x ,Y- D(X,Y) ~ (Y,S(X)) 

for each X G C r and Y € C; the isomorphism 4>x,Y is required to be natural both in X 
and Y. In fact, the functor S is an R-lmeai equivalence of categories; see Appendix 
A.l. We will call the functor S the generalized Serre functor on C, and we refer to C r 
(resp. Q) as the domain (resp. the range) of the generalized Serre functor. We will 
call this set of data the generalized Serre duality on C. Observe that the category 
C has Serre duality in the sense of [7] if and only if C r = C = C\. For details, see 
Appendix. 

Here is our main theorem. 

Main Theorem. Let T be a Horn- finite Krull-Schmidt triangulated category. De- 
note by S: T r — > Ti its generalized Serre functor. Then both T r and l~i are thick 
triangulated subcategories of T ■ Moreover, we have 

(1) there is a natural isomorphism rjx '■ S(X\V\) — > S(X)[1] for each X € T r such 
that the pair (S, if) is a triangle equivalence between T r and Ti; 

(2) an indecomposable object X inT belongs to Ti (resp. T r ) if and only if there 
is an Auslander-Reiten triangle in T containing X as the first term (resp. 
the third term). 
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Let us remark that the statement (1) is a generalization of a result due to Bon- 
dal and Kapranov ([7]; also see [61 [IS]). The main theorem is a combination of 
Propositions [231 [273 ESI and lATTl 

The paper is organized as follows. In section 2, we divide the proof of the main 
theorem into proving several propositions. Section 3 is devoted to computing the 
generalized Serre duality for the bounded derived categories of certain abelian cate- 
gories explicitly; see Theorem 13.51 and Theorem 13.101 The computational results are 
based on results by Happel (p3]), and by de Naeghel and Van den Bergh (|24J). We 
obtain, as a byproduct, a seemingly new characterization of Gorenstein algebras: an 
artin algebra A is Gorenstein if and only if the bounded homotopy category of finitely 
generated projective A-modules has Serre duality; see Corollary 13.91 The appendix 
explains the basic notions and results on generalized Serre duality on a Hom-finite 
linear category. 

For triangulated categories, we refer to [12\ [To"! 123] . For the notion of Auslander- 
Reiten triangles, we refer to [T21 H3]. 

2. Proof of Main Theorem 

In this section, we divide the proof of the main theorem into proving several propo- 
sitions. We make preparation on a study of coherent functors on triangulated cate- 
gories. We collect the relevant results which seems to be scattered in the literature. 

Let C be an additive category. Denote by (C op ,Ab) the large category of con- 
travariant additive functors from C to the category Ab of abelian groups. Note that 
although (C op , Ab) is usually not a category, but it still makes sense to mention the 
exact sequences of functors, in particular, the notions of kernel, cokernel and exten- 
sion in (C op , Ab) make sense. Hence sometimes we even pretend that (C op , Ab) is an 
abelian category. 

Recall that the Yoneda embedding functor p: C — > (C op , Ab) is defined by p(C) = 
(— ,C), where (— , C) denotes the representable functor Homc(— , C). Then Yoneda 
Lemma implies that p is fully faithful; moreover, representable functors are projective 
objects in (C op ,Ab). Recall that a functor F G (C op ,Ab) is said to be coherent 
provided that there exists an exact sequence of functors 

(- Co)— Ci)— ►(), 

where Cq,C\ € C. Note that such a sequence can be viewed as a projective presen- 
tation of F in (C op , Ab). Denote by C the full subcategory of (C op , Ab) consisting of 
coherent functors on C. Note that the category C of coherent functors is an additive 
category with small Horn sets. 

Recall that a pseudokernel of a morphism /: Cq —> C\ in C is a morphism 
k: K — > Co such that / o k = and that each morphism k! : K — > Cq with 
/ o k' = factors through k, in other words, the morphism k makes the sequence 

(—,K) (— , Co) (— ,Ci) of functors exact. We say that the category C has 
pseudokernels provided that each morphism has a pseudokernel. Dually, one defines 
pseudocokernels. 

Let us recall a basic result on coherent functors ( [21 [3] ; also see [19]). 

Lemma 2.1. Let C be an additive category. Then the category C of coherent functors 
is closed under cokernels and extensions in (C op , Ab). In particular, C is closed under 
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taking direct summands. Moreover, C is an abelian subcategory of (C op , Ab) if and 
only if C has pseudokernels. □ 

The following is well known, as pointed out in [19\ Example 4.1(2)]. 

Lemma 2.2. Let T be a triangulated category. Then T has pseudokernels and pseu- 
docokernels. 

Proof. Given a morphism v : Y — > Z in T, we take a distinguished triangle 

iAyAzAi[i]. 

Since for any object C £ T, the functors (C, — ) and (— , C) are cohomological (see 
[15j p.23]), it is easily verified that u and w are a pseudokernel and a pseudocokernel 
of respectively. □ 

In what follows T will be a triangulated category. We will introduce a two-sided 
resolution for any coherent functor on T ■ Given a functor F € T, we have a presen- 
tation 

(-.y)^-, Z)— >F— >0. 

Take a distinguished triangle X — Y Y — > Z ^> X[l]. So we have a long exact sequence 
of functors 

- — (-,*[-!]) ( ^ 1D (",*) (-.7) <^ (-,Z) U> (-,*[!]) — ... 
Note that .F ~ Coker(— , d). Hence we have the following two exact sequences 

- — {-,Z\-l\) ( -'^ lD (-,X) (-,Y) ^ (-, Z) — > F — > 0, 

and 

o — > f — > (-,*[!]) ( -,y[i]) ( ^ ]) ( -,z[i]) i-,X[2]) ••• 



Denote by Coho(T) the full subcategory of (7~ op , Ab) consisting of cohomological 
functors. By Lemmas 12.11 and 12.21 we deduce that the category T is an abelian 
category which has enough projective objects. In particular, we may define the 
extension groups Ext*(.F, G) for any coherent functors F and G, i > 1. 

The next lemma is also known; compare [231 p. 258]. 

Lemma 2.3. Let H ef. Then we have H G Coho(T) if and only if Ext* (F, H) = 
for all i > 1 and F £ T. 

Proof. Assume that F € T. Note that we just obtained a projective resolution for F. 
We take the resolution to compute Ext l (-F, H). Apply Yoneda Lemma ((— , C),H) ~ 
H{C). We obtain that Ext l (F, H) is just the i-th cohomology group of the following 
complex 

— >■ — ► fl"(y) — ^ H(X) —> H(Z[-1}) — 

Here we use the notation as in the above two-sided resolution. Now if H is coho- 
mological, the above complex is exact at all the positions other than the zeroth one 
(where H(Z) sits). Hence Ext*(F,H) = for all i > 1. 

On the other hand, assume that Ext l (F, H) = for all i > 1 and F € T. To see 
that H is cohomological, take a distinguished triangle I 4 7 A 2 4 Take 



GENERALIZED SERRE DUALITY 



5 



F = Cok(— ,v); it is a coherent functor. As in the above discussion, the assumption 
that Ext* (F, H) = will imply that the complex 

— >• H(Z) — > H(Y) — > H(X) — > H(Z[-1]) —>>••• 

is exact at all the positions other than the zeroth one. This implies that H is coho- 
mological. □ 

Denote by add(p(T)) the full subcategory of (T op ,Ab) consisting of direct sum- 
mands of representable functors. Observe that add(p(T)) C Coho(T) PI 7". 

The following result is due to Freyd and Verdier independently ([HI [29]). We refer 
to |23l p. 169] and |19|, 4.2] for a modern treatment. The result we present is in a 
slightly different form. Recall that an abelian category is Frobenius provided that 
it has enough projective and enough injective objects, and the class of projective 
objects coincides with the class of injective objects; see [121 Chapter 1.2]. 

Proposition 2.4. Let T be a triangulated category. Then the category T of coherent 
functors is a Frobenius abelian category such that the class of its projective-injective 
objects equals 

Coho(T) nf = add(p(T)). 

Proof. It is already observed that T C (T op , Ab) is an abelian subcategory which has 
enough projective objects. The class of projective objects equals add(p(T)). Note 
that (— , C) is a cohomological functor. By Lemma 12,3} we have, for each FgT and 
C € T, Ext*(F, (— j C)) = 0, i > 1. Therefore, the representable functor (— , C) is an 
injective object in T ■ Hence we have shown that T has enough projective objects 
and that every projective object is injective. 

We claim that T has enough injective objects and that every injective object is 
projective. To see this, assume that F is a coherent functor, and take a presentation 

(— ,Y) -V (— , Z) — > F — > 0. Then we have seen that there is a monomorphism 
F — > (—,X[1]) via the two-sided resolution of F. By the above proof, the functor 
(— ,A"[1]) is injective. Then the category T has enough injective objects. Assume 
further that F is an injective object in T ■ Then the monomorphism F — > (— ,X[1]) 
splits, and hence F is projective. This proves the claim, and then we have shown that 
T is a Frobenius category and the class of its projective-injective equals add(p(T)). 

It remains to show Coho(T) PI T = add(p(T)). We have already noticed that 
add(p(T)) C Coho(T) n f. Let H e Coho(T) P f. By Lemma [231 Ext i (F, H) = 
for all F € T and i > 0. Then H is an injective object in T, and hence by above H 
lies in add(p(T)). □ 

In what follows R will be a commutative artinian ring. We denote by -R-mod the 
category of finitely generated -R-modules. The triangulated category T will be R- 
linear which is Horn-finite and Krull-Schmidt. Consider the category (T op , -R-mod) 
of R- linear functors from T to i?-mod. Observe that (T op , -R-mod) is viewed as a full 
subcategory of (T op ,Ab) via the forgetful functor. Recall the notion of generalized 
Serre duality from the introduction. 

Proposition 2.5. The full subcategories T r and T\ are thick triangulated subcategories 
ofT. 
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Proof. We only prove the result on %.. We first claim that 

% = {X G T I D(X, -) G f }. 

To see this, note that T is Krull-Schmidt, hence it is idempotent-split; see [12] and 
[9l Theorem A.l]. Then we have add(p(7")) = p(T). Note that the functor D(X, — ) 
is always cohomological. Hence if D(X, — ) is coherent, then by Proposition 12. 4| 
D(X,-) G Coho(T) nf = add(p(T)) = p(T), that is, D(X,-) is representable. 
Hence, the functor D(X, — ) is coherent if and only if it is representable. Now the 
claim follows. 

Note that D(X[n], —) ~ D(X,—) o [— n]; here for each integer n, [n] denotes the 
n-th power of the translation functor [1] on T ■ Then the subcategory T r C T is closed 
under the functors [n]. Let X' — > X — >■ X" — s> A'[l] be a distinguished triangle such 
that X',X" G 7^. Then we have an exact sequence of functors 

(X'[l], -) — ► (X", -) — ► (X, -) — ► (X', -) — > (X"[-l], -), 

and hence the following sequence 

-) — ► D(X', -) — ► £>(X, -) — ► -) — ► £>(X'[1], -) 

is also exact. Since X',X" G %, the four terms other than D(X, — ) in the above 
sequence lie in T. By Lemmas 12.11 and 12.21 T C (T op , Ab) is an abelian subcategory 
which is closed under extensions. Hence we infer that D(X, — ) G T, and then by the 
claim above, we have X G T r . We have shown that T r is a triangulated subcategory 
of T ■ Observe that % is thick, that is, it is closed under taking direct summands, 
since direct summands of a coherent functor are still coherent. □ 

Let us assume that C is a Horn-finite i?-linear additive category which is skeletally 
small, that is, the iso-classes of objects in C form a set. Consider the full subcategory 
(C op ,i?-mod) of (C op ,Ab) consisting of i?-linear functors. Then we have duality 
functors 

D: (C op ,i?-mod) — ► (C,R-mod) and D : {C op , R-mod) — ► (C,i?-mod), 

which are induced by the Matlis duality on i?-mod. The additive category C is called 
a dualizing R-variety if the two duality functors D preserve coherent functors ([!]). 
It is easy to see that C is a dualizing R- variety if and only if C has pseudokernels and 
pseudocokernels, and D(X,—) G C and D(—,X) G C op for each object X G C; one 
may apply Lemma |2. II and [U Theorem 2.4(1)]. 

Recall from Appendix A.l that a Horn- finite category C is said to have Serre duality, 
if C r = C = Cj. Combining Lemma 12.21 with the claim (and its dual) in the above 
proof, we get the following observation; compare [17\ Proposition 2.11]. 

Corollary 2.6. Let T be a skeletally small Horn-finite Krull-Schmidt triangulated 
category. Then T has Serre duality if and only if T is a dualizing R-variety. □ 

The following result generalizes slightly a result due to Bondal and Kapranov (0 
Proposition 3.3]). Let us stress that the tricky proof is modified from an argument 
independently due to Huybrechts ([161 Proposition 1.46]) and Van den Bergh ([6, 
Theorem A.4.4]). Recall the notions of generalized Serre duality and trace function 
in the appendix. For the notion of triangle functor, we refer to |18t section 8]. 
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Proposition 2.7. Let T a Horn-finite Krull- Schmidt triangulated category. Then 
there is a natural isomorphism r/x- S(X[1]) — > S(X)[1] for each X € T r such that 
the pair (S, n) is a triangle functor between the triangulated categories % and T\. 

Proof. Recall from Proposition 12.51 that both T r and 7/ are triangulated categories of 
T. We will construct a natural isomorphism 

n x :S(X[l])^S(X)[l} 

for each X € T r , such that the pair (S,rj) is a triangle functor. 

Let X 6 T r and Y £ T. Consider the following composite of natural i?-linear 
isomorphisms 

^ X y. (Y,S(X[1])) D(X[1],Y)' D -^ 1] D(X,Y[-1]) 

Oy^x {Y [-l],S(X)) -B> (Y,S(X)[1}), 

where (ft is the natural isomorphism in Proposition IA.11 Note the minus sign in the 
second isomorphism above. By following the composition carefully, we obtain that, 
for each / € (X,Y[-1\) and g G (Y,S(X[1\)), 

(/, Vx,Y(g)[-l])x,Yl-l] = "(/[I], 9)X[1],Y- 

Note that \& defines an isomorphism ^x- (~t S(X[1])) ~ (—,S(X)[1]) of i?-linear 
functors. By Yoneda Lemma there is a unique isomorphism ijx '■ S(X[1]) S(X)[1] 
such that ^x = (—,Vx), in other words, ^x,Y{g) = VX 9- Consequently, we have 

(2-1) {f,(jlxog)[-l])x,Y[-i)=-(f[l],g)x[i},Y. 

We observe that ^x,Y is natural both in X and Y. It follows then that rjx is natural 
in X. In other words, the isomorphisms r/x define a natural isomorphism of functors. 

Next we show that the pair (S, n) is a triangle functor. Given a distinguished 
triangle I 4 7 4 Z 4 I[l] in T r . We have to show that the following triangle is 
also distinguished (in 7J, or in T) 

S {X) S M S{Y) S M S(Z) vx ^i w) S(X)[1}. 

We claim that the following sequence of functors is exact 
(-,S(Y)) — > (-S(Z)) — ► (-S(X)[1]) — > (-,S(Y)[l]) — ► (-S(Z)[1]). 

In fact, from the distinguished triangle X — >■ Y~ — > Z — Y X[l], we infer that the 
following sequence is exact 

(z[i], -) — ► (y[i], -) — ► (x[i], -) — ► (z, -) — ► (y, -), 

and hence the following sequence is also exact 

D(Y, -) — ► -) — ► £>(X[1], -) — > D(y[l], -) — ► D(Z[1], -). 

Note that in the following commutative diagram of functors all vertical morphisms 
are isomorphisms 

D(Y, -) D(Z, -) D(X[1], -) D(Y[1], -) D(Z[1], -) 



<i>Y 



<t>z 



^X°4>X\1] 



<B z od 



(-, S(Y)) (-, S{Z)) > (-, S(X)[lj) ^ (-, S(Y)[1}) ^ (-, S(Z)[1]). 
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Here 4>x{i], 4>Y[i] an d 4>z[i] are the natural isomorphisms of functors induced by 4> in 
Proposition IA.11 This proves the claim. 

Take a distinguished triangle S(X) S ^ S(Y) 4 W A S{X)[1}. We claim that to 
prove the result, it suffices to show that there is a morphism 5: W —> S{Z) such that 
the following diagram commutes. 



S (X)^ls{Y) 



S(X) 



S(u) 



S(Y) 



S(v) 



W 



S(Z) 



■n x oS(w) 



S(X)[1] 



S(X)[1] 



In fact, assume that we already have the above commutative diagram. Then we have 
the following commutative diagram of functors 



,S(X)) 



i-,S(X)) 



-,S(Y)) 



-,S(Y)) 



,W)- 

(-,5) 



(-,S(X)[1}) 



(-,S(X)[1}) 



-,S(Y)[1}) 



;S(Y)[1]) 



Note the the rows are exact (see the previous claim). Then Five Lemma implies that 
(—,5) is an isomorphism, and hence by Yoneda Lemma 5 is an isomorphism. We are 
done. 

To complete the proof, it suffices to find a required morphism 5. To find such a 
morphism 5 G (W, S(Z)) is to solve the two equations 5oa = S(v) and r]x°S(w)o5 = 
P. By (2.1) we have 

(2.2) Tr x {(r, x o /)[-!]) = -Tr x[1] (/), / G (X[l], S(X[1})). 

For the definition of Tr and the bilinear form (—,—), see Appendix. By the non- 
degeneratedness of the bilinear form (—,—), we infer that 5 o a = S(v) is equivalent 
to the equations Trz(S(v) o x) = Tr^(5 o a o x) for all x € (Z,S(Y)); and that 
r/x o S(w) o 5 = (3 is equivalent to the equations Trx((^x ° S(w) o 5)[— 1] o y) = 
Tr x ((3[-1] o y) for all y G (X, W[-l\). 
Note that we have 

Tr x ((vx o S(w) o «y)[-l] o y) = -Tr x[1] (S(w) o 6 o y [l}) 

= -Tr z (5 o y[l] o w), 

where the first equality uses (2.2) and the second uses (A. 4). Hence it suffices to find 
5 G (W,S(Z)) such that 

Tr z {5 oaox) = Tr z {S{v) ox), V x G (Z, S(Y)), 
Tr z (5 o y[l] o w ) = -Tr x (/3[-l] oy), V y G (X, W[-l\). 

By Proposition IA.ll and its proof we have the isomorphism 4>w,z- (W,S(Z)) ~ 
D(Z,W) and the equality 4>w,z{^) = Tr^(5 o — ). Here we recall that E is the 
minimal injective cogenerator for R and that D = Hom^(— ,E). Hence to complete 
the proof, it suffices to find an i?-linear morphism F from (Z, W) to E such that 

F(aox) = Tr z (S{v)ox), V x G (Z,S(Y)), 
F(y[l] o w ) = -Tr x (/3[-l] oy), V y G (X, W[-l]). 
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Using the injectivity of E, it is not hard to see that such a morphism F exists provided 
that whenever aox = y[l]ow, then Tiz(S(v) ox) = — Trx(/3[— 1] oy). Now we assume 
that aox = y[l]ow. Then we have the following morphism of distinguished triangles. 

Y Z — ^ x[l] — y [i] 



S(u) 



y[i] l <hW 



S(X) -U- S(Y) — ^ w 5(x)[l] 
Therefore we have 

Tr^(5(u) o x) = Try(a; o v) 

= Tr Y (S(u) o (p ) 
= TrxOo ° u) 
= Tr x (-P[-1] o y) 
= -Tr x (/3[-l]oy), 

where the first and third equality use (A. 4) and the second and fourth use the com- 
mutativity of the diagram above. This completes the proof. □ 

Let T be a Horn-finite Krull-Schmidt triangulated category as above. Recall from 
|12] and |13l 1.2] that an Auslander-Reiten triangle in T is a distinguished triangle 
I 4 7 4 2 4 such that both X and Z are indecomposable, and that w ^ 
satisfies that for each non-retraction 7: Y' — > Z in T we have w o 7=0. By |12|. 
p. 31] the morphism u is left almost-split, that is, any non-section h: X — > Y' factors 
through u; dually, the morphism v is right almost-split. 

The following result is a slight generalization of a result due to Reiten and Van 
den Bergh ( [2b\ Proposition 1.2.3]), where the result is formulated in a different 
terminology. 

Proposition 2.8. Let T be a Horn-finite Krull-Schmidt triangulated category, and 
let X G T be an indecomposable object. Then we have 

(1) X £ % if and only if there is an Auslander-Reiten triangle with X as the 
third term; 

(2) X G 7~i if and only if there is an Auslander-Reiten triangle with X as the first 
term. 

Proof. We only prove the first statement. By a dual argument, one proves the second 
one. 

For the "only if part, assume that X € % is indecomposable. Recall from Propo- 
sition [ATT] that the generalized Serre functor S : T r T\ is an equivalence. Therefore 
S(X) is also indecomposable. For each X € % and Y G T there is a non-degenerated 
bilinear form (— , —)x,Y '■ (A, Y) x (Y, S(X)) — > E; see the proof of Proposition IA.11 
Take a nonzero morphism w: X — > S(X) such that (rad Endj-fA), w)xx = 0. Here 
rad End-7-(A) denotes the Jacobson radical of End7-(A). Then we form a distin- 
guished triangle S(X)[-l] 47Al4 S(X). We claim that it is an Auslander- 
Reiten triangle. Then we are done. 

In fact, it suffices to show that each morphism 7: X' — > A, which is not split- 
epic, satisfies 11)07 = 0. To see this, for any x G (A, A'), consider (x,w o j)xx' = 
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(7 o x,w)x,x'i see (A. 2). Since 7 is a not split-epic, then 70x6 rad Enckr(X), and 
thus (7 o x, w)x,x = 0. Therefore (x, w o "f)x,X' = 0- By the non-degeneratedness of 
the bilinear form, we have w o 7 = 0. 

For the "if part, assume that we have an Auslander-Reiten triangle Z 4 7 A 
X — >■ Z[l]. Take any i?-linear morphism Tr: (X, Z[l]) — > E such that Tr(u>) 7^ 0. 
Here we use that E is an injective cogenerator. We claim that for each I'gT the 
pairing 

(-,-): (X,X')x(X',Z[l])^E 

given by (/, g) = Tr(po/) is non-degenerated. If so, we have an induced isomorphism 
of R- linear functors D(X, — ) ~ (— , Z[l]). Hence X £ and then we are done. 
To prove the claim, let X' € T. Let /: X — > X' be a nonzero morphism, and 

let X" A X -U X' -> X"[l] be a distinguished triangle. Since / / 0, then s is not 
split-epic. Then by the properties of Auslander-Reiten triangle, we have w o s = 0. 
This implies that w factors through /, that is, there is a morphism g: X' — >■ Z[l] 
such that w = go /, and hence (/, (7) 7^ 0. On the other hand, let g: X' — > Z[l] be a 

nonzero morphism, and let Z A- A""' — > X' A Z[l] be a distinguished triangle. Since 
5^0, then u' is not split-mono. Then by the properties of Auslander-Reiten triangle, 
we deduce that u' factors thorough u, and then we have the following morphism of 
distinguished triangles 



Z 



1 
1 

v 

X'" 



-X- 

1 

1/ 
y 

X' 



Z[l] 



Z 



1] 



Hence g o f = w and then (/, g) ^ 0. This completes the proof. 



□ 



3. Computing generalized Serre duality 

In this section, we will compute explicitly the generalized Serre duality for the 
bounded derived categories of artin algebras and of certain noncommutative projec- 
tive schemes in the sense of Artin and Zhang ( pQ ) ; see Theorem 13.51 and Theorem 

Emu 

Let R be a commutative artinian ring. Let C be an i?-linear category which is Horn- 
finite and which is assumed to be additive. Denote by Sc ■ C r — > Ci its generalized 
Serre functor. Denote by (— , —)x,Y '■ (X, Y) x (Y, Sc(X)) — > E the associated bilinear 
form for each X G C r and Y € C. Here E is the minimal injective cogenerator of R. 
For details, see Appendix. 

Let K fe (C) be the bounded homotopy category of complexes in C. Note that the 
category K b (C) is naturally R- linear and then is Horn-finite. Denote its general- 
ized Serre functor by 5: K. b (C) r —¥ K b (C)/. The following result is inspired by the 
argument in [TJJ p. 37]. 

Proposition 3.1. Use the notation as above. Then we have 

K b (C r ) C K b (C) r , K b (d) C K b (C) z and S\ Kb(Cr) = S c , 
where Sc is viewed as a functor on complexes between K 6 (C r ) and K fe (C/). 
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Proof. To prove the result, it suffices to build up a natural non-degenerated bilinear 
form 

(-,-) x . iY . : Hom K6(c) (X',y) x Hom K6(c) (Y' , S C (X')) — ► E 
for each X* £ K b (C r ) and Y* € K b (C). We define 

where /* = (/') and g* = (g l ) are chain morphisms. 

Note that the bilinear form is well defined. In fact, if /* is homotopic to 0, then 
(f',9')x',Y' = 0. Assume that p = d^ 1 o h* + h i+1 o d x for each i e Z. Here 
{h % : X % — >■ F* _1 }j g z; is the homotopy. Then we have 

(/", <7*)x.,y. = ^ <f W< + ("l)*^ 1 4, 9%^ 

= 0. 

The second equality follows from the identities (A.l) and (A. 2) in the appendix, and 
the last one follows from the fact that g* : Y* — >■ Sc(X') is a chain morphism. 

To prove the naturalness of the bilinear form is to verify the following two identities 

(foe% g')x>;v = (r, S c (9')og») x . tY . 

(7*°/*, 9*)x',v = (/*, j'oT'k'.v- 

Note that they can be easily derived from the identities (A.l) and (A. 2) for the 
category C. 

We will prove the non-degeneratedness. For this end, fix a complex X* E K 6 (C r ). 
Denote by 

4> : DBom Kb(c) (X', -) — ► Hom K 6 (c) (-, S C (X')) 

the natural transformation induced from the above bilinear form. We claim that (j) 
is a natural isomorphism. Then we are done. 

In fact, since 4> is a natural transformation between two cohomological functors, 
so by devissage, it suffices to prove that 

4> Y . : DRom Kb(c) (X' ,Y') — ► Rom Kb(c) (Y',S c (X-)) 

is an isomorphism for each stalk complex Y* . Without loss of generality, assume 
that Y* = Y for some object Y <G C. Then DRom K b^(X' ,Y) is just the zeroth 
cohomology of the following complex 

► D(X~ 1 ,Y) — > D(X°, Y) — > D{X\Y) — > ■ ■ ■ 

By the generalized Serre duality of C, the above complex isomorphic to the following 
complex 

► (Y^dX- 1 )) — ► (Y,,S C (A )) — > (Y^A 1 )) — ► ■■■ 

While the zeroth cohomology of this complex is nothing but Hom K &( C )(Y, Sc(X')). 
This proves that (fry is an isomorphism. □ 
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Let A be an i?-linear abelian category. Denote by Y) b (A) its bounded derived 
category, which is naturally i?-linear. We will assume that D b (A) is Horn-finite. 
In this case the abelian category A is said to be Ext- finite ([20]). Observe that an 
Ext-finite abelian category is Horn-finite. By [8j Corollary 2.10] the category Y) b (A) 
is idempotent-split, hence by [9j Theorem A. 2] it is Krull-Schmidt; also see |20t 
Corollary B]. Denote by V and 1 the full subcategory of A consisting of projective 
objects and injective objects, respectively. Let S A : A r — > A\ be the generalized Serre 
functor on A. 

We claim that A r Q V and Ai C X. Take an object X G At. By the generalized 
Serre duality, we have an isomorphism of functors (X,—) ~ D{— , S A (X)). Hence 
the functor (— , X) is right exact, and then the object X is projective. Similarly each 
object in A\ is injective. 

Note that we have natural isomorphisms 

Bom Db{A) (P',X') ~ Rom Kb{A) (P\X'), Rom Db(A) (X\ L') ~ Rom Kb{A) (X' , P), 

for any P* G K b (V), X' G K b (A) and P G K b (l). In particular, K b (V) and K b (l) 
are naturally viewed as triangulated subcategories of D b (A); compare 1.4] and 
[THl Example 12.2]. 

We have the following direct consequence of Proposition 13.11 

Corollary 3.2. Let A be an Ext-finite abelian category. Then we have 
K b (A r ) C V b (A) r , K b (Ai) C D b (A)i and S\ Kb{Ar) = S A , 
where S denotes the generalized Serre functor on D b (A). □ 

Following |24} Appendix] a complex X' in D b (A) is said to have finite projective 
dimension, if there exists some integer N such that Hom D 6^ (X*, M[n]) = for any 
M G A and n > N. Set D b (A)i p d to be the full subcategory of T) b (A) consisting of 
complexes of finite projective dimension; it is a thick triangulated subcategory. 

The following is well known; compare the proof of [27} Proposition 6.2]. 

Lemma 3.3. We have K. b (P) C D b (A)f p d- On the other hand, if the category A has 
enough projective objects, then K. b (V) = D 6 (^4)f p d. □ 

Dually one defines a complex of finite injective dimension and introduces the tri- 
angulated subcategory Y) b (A)M of Y) b (A). Then one has the dual version of Lemma 
13.31 compare [15j Chapter I, Proposition 7.6]. 

We have the following observation. 

Proposition 3.4. Let A be an Ext-finite abelian category. Then we have D b (A) r C 
D V) fpd and DV), C B b (A) M . 

Proof. Take a complex X' G B b (A) r . Then we have an isomorphism of functors 
over Y) b (A): Hom D b^(X', — ) ~ DRom- Db ^(—,S(X')), where S is the generalized 
Serre functor on T) b (A). Take a positive integer N such that H n (S(X')) = for any 
n < —N; here H n (—) denotes the n-th cohomology of a complex. Let M G A and 
let n > N. Then Hom Dk(<4) {X*,M[n\) ~ DHom Db(x) (M[n\, 5(X')) = 0. Hence X* 
has finite projective dimension. Then we get the first inclusion, and the second one 
can be proved dually. □ 
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Here is our first explicit result on generalized Serre duality. 

Theorem 3.5. Let A be a Horn-finite abelian category with enough projective and 
enough injective objects. Assume that A r = V and Ai = X. Denote the generalized 
Serre functor S A by v which is called the Nakayama functor. Then we have D b (A) r = 
K. b (P), D b (A)i = K b (Z) and the generalized Serre functor on D b (A) is given by v 
which operates on complexes term by term. 

Proof. Note that under the assumption, the abelian category A is Ext-finite. Then 
the result follows immediately from Corollary 13. 2\ Lemma 13.31 and Proposition 13.41 

□ 

We have the following motivating example of Theorem 13.51 

Example 3.6. (Happel) Let A be an artin R-algebra. Denote by j4-mod the category 
of finitely generated left A-modules. We denote by A-proj (resp. A-m.}) the full 
subcategory consisting of projective (resp. injective) modules. 

It is well known that the category j4-mod satisfies the assumption of Theorem 
\3.5l In this case, the functor v is the usual Nakayama functor DHom^— , ^4); see 
\V2\ p. 37]. We apply Theorem 13.51 to conclude that D 6 (^4-mod) r = K fe (^4-proj), 
D 6 (j4-mod)i = K 6 (j4-inj) and the generalized Serre functor is given by v. We apply 
this explicit computational result. We infer from the statement (2) in Main Theorem 
that in an Auslander-Reiten triangle X' —> Y* — > Z* — >■ A*[l] in D 6 (^4-mod) we 
have X* £ K b (^4-inj) and Z* G K b (>l-proj). Moreover, we have X* ~ u(Z*); see the 
proof of Proposition [Ql This observation is essentially due to Happel ([13, Theorem 
1.4]). ' 

In what follows we will consider the conditions (C) and (C) which are introduced 
in Appendix A. 2. We have the following observation. 

Lemma 3.7. Let A be a Horn-finite abelian category. Then we have 

(1) if A has enough projective objects and A r = V, then D b (A) satisfies the 
condition ( C); 

(2) if A has enough injective objects and Ai =T, then D b (A) satisfies the condi- 
tion (C). 

Proof. We only prove (1). Assume that A has enough projective objects and that 
A r = V. By Corollary^ LemmaE3]and Proposition^! we have T> b {A) r = K b (V). 

Consider complexes X',X*' € D b (A) r and Z' E D b (A). Since A has enough 
projective objects, we may take a quasi-isomorphism P' — > Z* such that P' is a 
bounded-above complex of projective objects with finitely many cohomologies; see 
[T5] Chapter I, Lemma 4.6]. Let n » 0. Take Z" = a^- n P m to be the brutal 
truncation, and take s to be the following composite 

Z" — ► P* — ► Z 9 , 

where Z* — > P* is the natural chain map. Observe that Z" G D b (^4) r . It is direct 
to check that the following two maps induced by s 

Rom mA) (X',Z") Kom mA) (X',Z'), Hom D6( ^(Z", X") Hom Db( ^ {Z\ X") 



are isomorphisms. We are done with the condition (C). 



□ 
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We will call an abelian category A a Gorenstein category provided that T) h {A)i v< i 
coincides with T) b (A)^d] compare [2]. By combining Theorem 13.51 Lemma 13.71 and 
Proposition IA.4I together, we have the following result. 

Proposition 3.8. Let A be a Horn-finite abelian category with enough projective and 
enough injective objects. Assume that A r = V and Ai = T. Then the following 
statements are equivalent: 

(1) the category K. b (V) has Serre duality; 

(2) the abelian category A is Gorenstein; 

(3) the category K fe (X) has Serre duality. □ 

The following immediate consequence gives a characterization of Gorenstein alge- 
bras, which seems to be of independent interest. Recall that an artin i?-algebra A 
is Gorenstein provided that the regular module A has finite injective dimension on 
both sides (|14J). We observe that an artin algebra A is Gorenstein if and only if 
the abelian category A-mod of finitely generated left ^-modules is Gorenstein in the 
sense above. 

Corollary 3.9. (compare |14|, Theorem 3.4]) Let A be an artin R-algebra. Then the 
following statements are equivalent: 

(1) the category K. b (A-pro}) has Serre duality; 

(2) the artin algebra A is Gorenstein; 

(3) the category K 6 (A-inj) has Serre duality. □ 



We will next consider the generalized Serre duality on the bounded derived category 
of coherent sheaves on certain noncommutative projective schemes in the sense of 
Artin and Zhang ([I]). 

We will follow |24[ section 2 and Appendix] closely. Recall some standard notation. 
Let A; be a field. Let A = © n >o^4n be a positively graded /c-algebra which is connected, 
that is, Aq ~ k. We will assume that A is left noetherian. Denote by A-Gr (resp. 
A-gr) the category of graded left A-modules (resp. finitely-generated graded left 
^-modules) with morphisms preserving degrees. For each d G Z, denote by (d) be 
the degree-shift functor on ^4-Gr and ^4-gr. For j > 0, denote by Ext^_ Gr (— , — ) 
and Ext J (— , — ) the Ext functors on the category A-Gr and the associated graded 
Ext functors, respectively. Then we have Ext 3 ' (M, N) = deZ Ext^_ Gr (M, N(d)) for 
graded modules M, N. We will view k as a graded ^4-module concentrated at degree 
zero. 

For a graded A- module M denote by r(M) the sum of all its finite dimensional 
graded submodules. This gives rise naturally to the torsion functor r: ^4-Gr — > A-Gi. 
A graded module M is called a torsion module provided that r(M) = M. Denote by 
A- Tor (resp. ^4-tor) the full subcategories of A-Gr (resp. A-gr) consisting of torsion 
modules. Note that both of them are Serre subcategories. One defines the quotient 
abelian categories 

Tails(A) = A-Gr/ A- Tor and tails(A) = A-gr/A-tor, 

which are called the category of quasi- coherent and coherent sheaves on the noncom- 
mutative projective scheme Proj(A), respectively. Observe that both categories are 
naturally /c-linear. 
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A connected left noetherian algebra A is said to satisfy the condition "x " provided 
that for each module M € ^4-gr, the Ext spaces Ext J (k, M) are finite dimensional for 
all j > 1; compare [H Definition 3.2]. In this case, the abelian category tails(-A) is 
Ext-finite; see [241 Proposition 2.2.1]. 

Here is our second explicit result on generalized Serre duality. Let us remark that 
it is somehow a restatement of a result due to de Naeghel and Van den Bergh ([Ml 
Theorem A. 4]); compare [28, Proposition 7.48] for the commutative case. For the 
notion of balanced dualizing complex and other unexplained notions, we refer to [30] 
and [23. 

Theorem 3.10. Let A be a connected (two-sided) noetherian algebra over a field k. 
Assume that 

(1) the algebra A satisfies "x" and the functor r is of finite cohomological dimen- 
sion; 

(2) the opposite algebra A op satisfies "x" and the functor r op is of finite cohomo- 
logical dimension. 

Then we have D b (tails(A)) r = Details (A) ) fpd; D b (tails(A))j = Details (A) ) fid and 
the generalized Serre functor is given S = — ® L 1Z*[— 1], where 1Z° is induced by the 
balanced dualizing complex R* of A. 

Proof. Note that the assumption implies the existence of a balanced dualizing com- 
plex R* in the sense of |30j . Here the functor — (g> L 1Z° is induced by the functor 
- <g> L R*. For details, see [211 Appendix]. 

Note that the abelian category tails(j4) is Ext-finite. By Proposition 13. 4| we have 

D fc (tails(,4)) r C D b (tails(A)) fpd and D b (tails(A))j C D fe (tails(^)) fid - 

We apply [21 Theorem A.4]. We obtain Details (A) ) r D D fe (tails(,4)) f p d ; moreover, 
the restriction of the generalized Serre functor 1 o b (tails (^4) ) fpd i s isomorphic to — <8> L 
1Z'[— 1], which is an equivalence between D b (tails(A))f pd and Details (A) )g d ; here we 
implicitly use [H Proposition 2.3] and [24, Lemma 2.2]. This equivalence implies that 
D fe (tails(A)) fid C Details (A) ) z , completing the proof. □ 

We end this section with two remarks. 

Remark 3.11. (1) Let T be a Horn-finite Krull-Schmidt triangulated category. We 
remark that the Verdier quotient triangulated categories T /%■ and T/Ti measure 
how far the triangulated category T is from having Serre duality. Observe that in 
the case of Example 13.61 the quotient category T '/%■ coincides with the singularity 
category of the artin algebra A ([23 ED]); similarly, in the (commutative) case of 
Theorem 13. 101 the quotient category T/T r coincides with the singularity category of 
the corresponding (commutative) projective scheme. 

(2) Note that there is an analogue of Reiten-Van den Bergh's theorem for abelian 
categories in |21j . Roughly speaking, a Horn-finite abelian category has the so-called 
Auslander-Reiten duality if and only if it has enough Auslander-Reiten sequences. We 
expect that there exists a reasonable notion of generalized Auslander-Reiten duality 
for an arbitrary Horn-finite abelian category. This generalized duality might be useful 
in the study of an abelian category which does not have enough Auslander-Reiten 
sequences. For example, the category of finite dimensional comodules over a coalgebra 
is often such an abelian category. 
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Appendix A. Generalized Serre duality on linear category 

Throughout this appendix, let R be a commutative artinian ring. Let C be an 
.R-linear category which is Horn-finite. We do not assume that C is an additive 
category. Use the notation as in the introduction. For example, for two objects A 
and Y, (A, Y) stands for Home(A, Y); D = Hom/j(— , E) is the Matlis duality, where 
E is the minimal injective cogenerator of R. 

A.l. In this subsection, we introduce the notion of generalized Serre duality on a 
linear category. One might compare with the treatment in Proposition 3.4], |26t 
p.301] and section 3]. 

Let C be a linear category as above. Recall the two full subcategories C r and Ci in 
the introduction. We have the following basic result. 

Proposition A.l. Use the notation as above. Then there is a unique functor (up 
to isomorphism) S : C r — )■ C\ such that for each X G C r and Y G C, there is a natural 
R-linear isomorphism 

4>xy: D(X,Y)~(Y,S(X)). 
Moreover, the functor S is an R-linear equivalence between C r and C[. 

Proof. We will first construct the functor S. Take an object X G C r . Then there 
is an object S(X) G C such that there is an isomorphism of .R-linear functors 
cpx '■ D(X, — ) ~ (— , S(X)). We stress that this isomorphism is required to preserve 
the .R-linear structure. 

Given any morphism /: X — > Y in C r , define the morphism S(f): S(X) — > S(Y) 
such that the following diagram commutes 

D(X,-)^(-,S(X)) 



D(Y,-)^(-,S(Y)) 

Here we apply Yoneda Lemma. Moreover, the morphism S(f) is uniquely determined 
by the above commutative diagram. It is direct to check that S : C r — > C is a .R-linear 
functor. Note that for each object X E C r , we have an isomorphism of .R-linear 
functors D(—,S(X)) ~ (A,—), and then we have S(X) G C\. In other words, we 
have obtained an R- linear functor S : C r —> C\. By the construction of S, we observe 
that there is a natural .R-linear isomorphism 

<I>x,y: D(X,Y) ^ (Y,S(X)) 

for X G C r and Y G C. Here we put 4>x,Y = <f>x(Y). The naturalness of <fix,Y in 
Y is obvious, while its naturalness in A is a direct consequence of the action of the 
functor S on morphisms. 

For the uniqueness of the functor S, assume that there is another functor S' : C r — > 
Ci such that there is a natural .R-linear isomorphism 4>' X y- ^O^j^O — O^^'i-^-))- 
In particular, there is an isomorphism of .R-linear functors D(X,—) ~ (— ,S'(X)). 
However by the above proof, we have seen (j>x '■ D(X,—) ~ (— ,S(X)). Therefore 
(— ,S(X)) ~ (— ,S"(A)), and hence Yoneda Lemma implies that there is a unique 
isomorphism S(X) ~ S'(X). It is routine to check that this gives rise to a natural 
isomorphism between the .R-linear functors S and S'. 
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We will show that the functor S: C r — > C\ is fully faithful and dense, and then 
it is a i?-linear equivalence of categories. For this end, we need to introduce some 
notation. For any X G C r and Y G C, define a non-degenerated bilinear form 

(-,-)x,y: (X,Y) x (Y,S{X))—>E 
such that (/, g)x,Y = ( t > x 1 Y(d)(f)- By the naturalness of (f>x,Y> we have 
(A.l) {fo9, g) X ',Y = {f, S(6)o g) X;Y 

(A. 2) (/, g o -y) x ,Y> = (l°f, g)x,Y, 

where 9: X' — ^ X and 7: Y' — > Y are arbitrary morphisms. Note that in (A.l), 
X,X' G C r , /: X -> Y and g: Y -> S(X); in (A.2), X G C r , /: X -> F' and 
5: Y^S(X). 

We will call the bilinear form (— , — ) defined above the associated bilinear form 
to S. Note that the following direct consequence of the above definition: F(f) = 
{f,<f>x,Y( F ))x,Y for each F G D(X,Y) and / G (X,Y). 

Let X, Y G C r . We claim that for each morphism / G (X, Y) and g G (Y, <S(X)) 
we have 

(A.3) (/, g)x,Y = (g, S(f)) Y>s(X ). 

In fact, we have 

(/, g)x,Y = (f ° Idx, g)x,Y use (A.l) 
= (ld x , S(f)og) x ,x use (A.2) 
= (5, S(f)) Y , S (x)- 
Consider the following composite of i?-linear isomorphisms 

$: (A,Y) D[4> £^ rl D{Y,S(X)) (S(X),S{Y)). 

Note that D^xx)' 1 (f)(9) = (f,g)x,Y and F(g) = (g, ^ y ,s(x)(F))y,s(x) all 
F G -D(Y, S(X)). Then it is direct to see that (f,g)x,Y = (<7, < ^ ) (/))y,5(x)- Compare 
this with (A.3), and note that the bilinear form is non-degenerated. We infer that 
S(f) = <&(/). Consequently the functor S is fully faithful. 

To see that S is dense, let X' G Ci. Note that the functor D(—,X') is representable. 
Assume that D(-,X') ~ (X,-). Then £>(X,-) ~ (-,X'), and this implies that 
X G C r . Therefore we have 

(r,S{X))'d D(X,-)*{-,J?). 

By Yoneda Lemma S(X) ~ X'. Then the functor 5 is dense, completing the proof. 

□ 

Following |26j we will introduce the notion of trace function. For each object 
X G C r , define its trace function to be an R- linear map 

Tr x : (X,S(X)) — > E 

such that Trx(f) = (ld x ,f)x,x. Then by (A.2), we have (f,g)x,Y = Ti x {s°f)- By 
(A.3), we get 

(A.4) Tr x (gof) = Tr Y (S(f)og). 
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Let C be a Horn-finite i?-linear category. We call the above obtained sextuple 

{S, C r , C t , <j), (-,-), Tr} 

the generalized Serre duality on the category C. The functor S is called the generalized 
Serre functor of C, where C r C C (resp. Ci C C) is referred as the domain (resp. the 
range) of the generalized Serre functor. 

We say that the category C has right Serre duality f[26j) provided that C r = C. In 
this case, we call S the right Serre functor of C. Similarly, we define the notion of 
having left Serre duality. The category C has Serre duality ([7J) provided that it has 
both right and left Serre duality. This is equivalent to C r = C = C\. 

A. 2. Let C be an i?-linear category which is Horn-finite. Let C r and Ci be the domain 
and the range of its generalized Serre functor, respectively. In this subsection we will 
consider the generalized Serre duality on both C r and C/. 

Consider the following two conditions on the category C: 

(C) for each A, X' G C r and Z G C, there exists a morphism s: Z' — > Z with 

Z' G C r such that (X, Z') ( ~ S) (X, Z) and (Z', X 1 ) ) (Z, A'); 
(C) for each Y,Y' G Ci and Z G C, there exists a morphism s: Z — >• Z' with 

Z' G such that (Y, Z) ( ~ S) (Y, Z') and (Z', F') ) (Z, Y"'). 
We have the following result, which is inspired by [144 Theorem 3.4]. 

Lemma A. 2. Let C be as above. Then we have 

(1) if Ci C C r , then (C r ) r = C r ; 

(2) conversely, if (C r ) r = C r and the category C satisfies the condition (C), then 
C\ C C r . 

Proof. (1) is obvious from the definition. 

To see (2), let X G C r . Since (C r ) r = C r , then we have an isomorphism of i?-linear 
functors on C r : D{X, —) ~ ( — , X'), where X' G C r . We claim that this isomorphism 
D(X, — ) ~ (— , X') can be extended on C. Then we get S(X) = X' , where S is the 
generalized Serre functor on C. By Proposition IA.ll we have Ci = S(C r ). Then we 
conclude that Q C C r . 

In fact, for each Z G C, take s: Z' — > Z to be the morphism in the condition (C). 
Then we have a sequence of isomorphisms 

D(X,s) , , , (s,X') 

D(X, Z) ~ D(X, Z') ~ (Z', A ) ~ (Z, A'). 

Note that this composite isomorphism is given by the trace function on A; more 
precisely, the inverse of this isomorphism is given by / i — > (g i-> Trx(/ ° g)); here 
Tr is the trace function given by the generalized Serre duality on C r . Then we get a 
natural isomorphism of i?-linear functors on C: D(X, — ) ~ ( — , A'). We are done. □ 

Dually, we have the following result. 

Lemma A. 3. Let C be as above. Then we have 

(1) ifC r C Ci, then = C\; 

(2) conversely, if (Cj)j = Ci and the category C satisfies the condition (C), then 
C r QCi. □ 

The following result seems to be of interest. 
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Proposition A. 4. Let C be a Horn-finite R-linear category. Let C r and Ci the domain 
and the range of the generalized Serre functor on C, respectively. Assume that the 
category C satisfies the conditions (C) and (C). Then the following statements are 
equivalent: 

(1) the category C r has Serre duality; 

(2) we have C r = Ci; 

(3) the category C\ has Serre duality. 

Proof. The implications (2) =4> (1) and (1) =>■ (3) follow from Lemma IA.2f 1) and 
Lemma IA,3f 1 ) , respectively. 

To see (1) => (2), we apply Lemma |A.2( 2) and then we get C\ C C r . Therefore 
(Ci)i 5 Ci n (C r )i. Here we use an easy fact that for any full subcategory 2? of a 
category C, one has Pj D Q fl V. However by (1), (C r )i = C r . Hence (Cj)j = C\. Now 
applying Lemma lA.3f 2). we get C r C C[. Then we are done. Similarly we prove the 
implication (2) =>- (3). □ 
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